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Confrontation with data that has been identified by school faculty as value 
indicatom of student life or of their life in the school conununity can support the 
unfreezing of the status quo necessaiy for mijor school changes. Repeated 
confrontation in the action research cycle with data transformed into information, 
fed into the decision-making and action-taking cycle, and then used to monitor the 
results of action taken, can support the moving and refreezing necessary to new 
directions and behaviors. Data collection and use, then, can serve as the gate from 
which collective actions for improvement emerge. 

When conducting action research, school faculties engage in the collection 
and analysis of data to help select and cUrify goals, select pioceduies, and study 
progress in terms of effects on students, teachers, administaators, and parents and 
other community members. One of the three major goals of the PSI League of 
Professional Schook is to involve schools in action researeh as part of the school- 
improvement process. The other two goals of the League are establishment of 
teacher involvement in schoolwide instructional decisions through a shared 
governance process and the development of school-improvement initiatives, 
especially ones focused on instruction. 

To study and evaluate goals is a published premise of the League itself and 
of each League school As part of their application for membership in the League, 
twoity-four schools signed a commitment to collect data to assess their progress 
on instructional initiatives. The premise is that developing accurate assessment of 
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ooUectivdy esUbUshed goals pushes i school faculty to conoq>tualize what is 
worth accomplishing thiough cuniculum, staff development, or instruction. 
Continuing study of the behavior, attitudes, and knowledge desired by the school 
community can darify emerging goahi and hdp shape plans toward optimum 
possibilities for school improvement. 

Another operational premise of the League stresses the value of having all 
actors in the school community (parents, students, teachers, adminisbators, and 
central oflke) engaged in information seeking and empirical problem solving 
aimed at making their world - from student life at the classroom Uble, to 
classroom instruction, to the school schedule - a healthier pUice, socially, 
intellectually, and physically. The value of modeling lifdong leuning and problem 
solving in the professional environment in which one has chosen to live supports 
personal, professional growth (Dewey, 1904; Gardner, 1963; CHoule, 1980; and 
Sdion, 1983) and provides continuous demonstrations of these criticd life 
processes for students. Or, simply stated, actions speak louder than words, what 
w«; do carries more wdght than what we say, what are the real "basics" educators 
wish to convey, or as Schaefer (1967) stated: 

How to induce more children to grapple zestfully with academic 
issues may dude our most determined efforts. But I strongly suspect 
that observing adults honestly wrestling with intellectual problems 
might win more youngsters to the life of the mind than any other 
experience the schools could devise, (p. 77) 



Data Use and Action Research 



Brief History nf Actinn RftWPJimti 

Data use within the League of Professional Schools matches an action- 
research-for-sodal-change framework. The origin of action research is usually 
attributed to Kurt Lewin (1947, 1948) and his students (e.g. Lippitt) although 
Corey (19S3) divides the honor between Lewin and John Collier. Collier was 
commissioner of Indian Affairs from 1933 to 1945; he felt that administrators and 
layman should partidpate in research , impelled "from their own areas of need 
because the problems are theirs and they must live with them" (Collier, 1945, in 
Corey, p. 7). Lewin's integration of action-taking into experimental social sdence 
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leseaich wu published in in "Action Reseirch tnd Minority Problems" and 
in 1947 in "Group Decision and Social Change." Both articles define action 
research as a three-step spiral process of 1) planning which involves 
"reconnaissance or fact-finding"; 2) taking actions; and 3) fact-finding about the 
results of the action (Lewin, 1947). "Planning usually starts with something like a 
general idea. For one reason or another it seems derirable to reach a certain 
objective. Exactly how to circumscribe this objective and how to reach it is 
frequently not too ckar." Thus, we reach intuitively toward general goals and 
means and gradually progress toward a plan specific enou^ to act on. 

Lewin (1947) bdieved that social problems should be served by social 
inquiry. His action research approach that applied the methodology of social 
science to immediate practical miyor problems developed out of his work to 
improve intergroup relations, such as vedudng anti-Semitism and racial prejudice. 
Lewin challenged the compartmentalization of research from action and the 
separation of research personnd firom active players. He advocated including 
practitioners from the arena under investigation in all phases of the research. He 
also challenged practitioners to use the tools of the social scientist to bring about 
change: "It seems to be crucial for the progress of social science that the 
practitioner understand that through social sciences and only through them can he 
gain the power necessary to do a good job" (Lewin, 1948, p. 213). 

Lewin and others who developed the action researoh concept emphasized 
collective rather than individualistic problem-solving and study. Thus, action 
researchers study problems which grow out of the community, work within a 
group to determine actions to be taken, and evaluate the effect of these actions 
within the community setting. Lewin advocated group work as part of the action 
research process because of the power of group discussion and interaction in 
producing commitment and because of the support for changes in individual 
attitudes and bdiavior provided by group interaction (1947). Chein, Cook, and 
Harding (1948) recommended practitioner involvement to build awareness of the 
need for the actions taken and greater personal investment in making the actions 
work. 

Corey (1953) was one of the first to officially promote action research in the 
field of educatioiL His definition of action researoh was the "process by which 
practitioners' attempt to study their problems scientifically in order to guide, 
correct, and evaluate their decisions and actions" (p. 6). His thesis was that school 
practitioners would make better decisions and implement more effective practices if 
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they Gondiictod reaeaxch u part of tlidr decision-inaldiig process and used the 
results of such research aa a ^de to selection or modification of their practice. 
The value of action research for Con^ was "determined hy the extent to which 
iindings lead to improvement in the practices of the people engaged in the 
research" (p. 13). Through involving teachers, administrators, and supervisors in 
studying their work (teaching) and in applying these findings to their school 
setting, changes would be more likdy to occur. 

Because of its emphasis on individual interactions within a group setting to 
support changes in behavior, its Qydes of fact-finding to feed decision making and 
to revise/evaluate actions, its use for htrge scale aofiial problems - and school 
change requires resociaUzation, its bent beyond acquisition of knowledge toward 
action steps within the context studied, and its opportunities for all actors within 
the school community to be engaged in group investigation and active problem 
solving to generate a better learning/living environment, action research has 
potential as one of the tools to be used in school improvement. This approach 
applied to data collection and use of site specific information is particularly 
compatible with school improvement through shared governance because shared 
governance requires group decision-making and uses collective action to bring 
about change in practice. Another operational premise of the League is that the 
group dynamics involved in schoolwide data collection, analysis, and use will serve 
to support change at the micro-level within the school and provide additional 
support for change at the macro-level within and througihout the League. 

The concepts of teacher-research and action-research for individual 
investigation of problems are completely compatible with this approach. However, 
the focus of action research as promoted through League meetings and support 
services emphasizes collective study of the school by its community of inhabitants. 
Each school creates its distinctive structure that involves various groups to 
different extents in particular activities. The range of persons involved is 
detcnnined by the school. Thus, as we will see in the following pages, teachers, 
administrators, parents, other community members, students, and support staff all 
may be involved, but the extent of participation varies by site. 
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Action RfaMroh Fnr Scliofll TmproYfimfiiit 

"Data of vaiying quality are pksntifiil. 
Undentanding is laie." 
(Alvin Tofiler, fifisfiabiO, 1990, p. 289.) 

Action research or data use to guide action is concqstually simple; it sounds, 
looks, and feels much like slightly foimdizcd problem solving, which it is. 
ReguUr application of this approach as a nonnal way of doing business, in making 
the school a center of inquiiy, however, is difficult to accomplish, (Sirotnik, 1987). 
Whether in schools (Goodlad, 1984) or in business and industry C^ppitt et al., 
198S), answers to "what has been accomplished through the changes hnplementedT 
are not readily available to most stakeholders. Lippitt cites two surv^ that 
address the lack of evaluation in planned organizational change in business and 
industry. Of 160 published reports of change interventions examined by Porras 
and Berg, only 20 included evaluation components to assess organizational or 
work group changes. It is interesting to examine the reasons. In a survey of 76 
consultants and clients, Bidwell and Lippitt (1971) found four major factors dted 
as obstacles to evaluation: Vuk of time, lack of criteria or a frame of reference, 
inability to develop measurable objectives, and hu;k of money. What is of interest 
to us is that these four factors are similar to the ones mentioned by League teams 
in relation to assessment of progress on their initiatives, especially time, criteria, 
and how do we measure "it." 

Lewin's work in the 1940*8 with a variety of organizations, from single 
schools to minority organizations to labor and management representatives has 
remarkable relevance today: 

Two basic facts emerged from these contacts: there 
exists a great amount of good-will, of readiness to face the problem 
squarely and really do something about it... [Yet] These eager people, 
feel themselves to be in the fog. They fed in the fog on three counts: 
1 . What is the present situation? 2. What are the dangers? 3. And 
most important of all, what shall we do? (1948, p.201) 

Lewin considered the lack of clarity about what ought to be done as one of 
the greatest obstacles to improvement of intergroup relations. He fdt that one of 
the consequences of this unclearness was the lack of standards by which to 
measure progress: 
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In a field that lacks objective standanls of achievemeat, no learning can take 
place. If we cannot judge whether an action has led forward or backward, if 
we have no criteria for evaluating the relation between effort and 
achievement, there is nothing to prevent us from making the wrong 
conclusions and to encourage the wrong work habits. Realistic fact-finding 
and evaluation is a pverfquisite for any learning (Lewin, 1948, p. 202). 

Lewin also identified another "sevens effect" of the inability to detennine 
the relationship between efforts made and achievement. This failure to measure the 
effects of actions designed to lead to improved conditions within an organization 
dq)rive8 workers [teachers, administrators, genend educators, and students] "of 
thdr legitimate desire for satisfaction on a realistic basis." Because descriptive 
fact-finding about educational innovations, regardless of simplicity, complexity, 
cost in money or time -from adopting a new handwriting program, to use of 
manipulatives K-8 for mathematics, to adopting a literature based approach to 
language arts,to block-schedules in the middle and high schools, to mainstreaming 
of students, etc.-is not an integral operating behavior in most school cultures, 
satisfaction or dissatisfaction with achievement relative to an innovation remains 
personalized and is mainly a matter of individual "temperament." Professional 
satisfaction for members of a dynamic organization requires r^iwtivff ^fi mytanoB 
of the continous cycle of 1) measuring progress and 2) taking action. As long as 
the determinination of success or failure of an innovation, even if it was originally 
selected by a team or a school council or the entire faculty, is left solely to the 
educators as individuals, collective action for school improvement or "site-based, 
collaborative school improvement" will be impossible. 



As school teams struggle to detennine how they will measure their school 
improvement goals, both the goals and the routes or actions to take in achieving 
them should become clearer. Effective schools demonstrate improved achievement 
over time and regulariy collect and use data to assess student performance 
(Glickman, 1990, p. 2S3). Yet, the action research process of identifying problem 
areas and ideas worth pursuing, gathering relevant data, discussing these data, 
formulating solutions, determining actions, and assessing the effects of these 
actions is a capacity "lacking in most schools" (Goodlad, 1984, p. 276). 
Fortunately, schooUi in the United States exist in a national culture that has long 
revered a pragmatic, problem solving approach to social action. Because of the 
support for this approach to change in the macro-system, the capacity for action 
research for school improvement could be rapidly developed within schools. 
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Actinn UmM^rrh within tlii» I#«gi.i> ^ f PrnfiftMrintial SrhtutU 



Through League activities, school teams are encouraged to develop the 
school as a center of inquiry: to ai>ply group investigation procedures as a way of 
life. The fonnal collection of data, followed by group analysis and interpretation, 
may move the school community forward in the path it has elected to follow. This 
regular confrontation with data as a progress maricer can function as "choice 
points" for the organization, as both "noticing choice" and "doing choice." 
Paraphrasing Wdck on the role of choice in the organizing process, the school 
community can ask itself "Knowing what we know now, should we naticB 
something we did not notice before and ignore something we noticed before?" and 
"Knowing what we know now, should we act differently?" From baseline data to 
regular checks on progress, educators face a series of choice points for unfreezing 
action and changing the experience of schooling. 

Shared gAvi>rn«tic« for its value in using the collective wisdom resident on- 
site in any school and the collective energy and resodalization needed to bring 
about major school change; the gniph«ri« nn inntnictinii for enhanced education 
because "teaching" is the major work of the school; and the drive for action 
r^^r^^ in order to know the current status of progress on goals, to make more 
informed, better decisions, to direct action to optimum possibilities for goal 
atta inmen t, to model this problem-solving approach to life as a normal way of 
business in schoohi both for the benefit of professionals living there and students 
required to be there - of these thiee m^^or thnuifai nf tha T^pue nf Pmfii«H«f,«ii 
Schools, shared governance has moved most swiftly into place with action researeh 
lagging far behind. 

Some of the factors appearing to inhibit ready application of action researeh 
in League schools are cited in business and organization (lippitt et al., 1985): 
time and methodology - (When do we do it? How do we do it?). Other factors 
seem to be a native connotation around use of test results. Possibly because of 
the history in our state associated with data collection through standardized 
testing in which tebts wep selected by "outsidere," data analyzed by "outsiders," 
and returned to schools as judgments about the work of teachers and prindpaL. 
"Date collector" was also, eaily on (late 1970*8), the name given to those who 
assessed beginning teachers with the n^nrgia t^^jipW , N arfninmnr^ AM>HHm»#>«» 
Tnatnimfflif. The term xsaausk and the methodology anc language associated with 
it has negative connotations for some practitioners (Reys Hl Yeager, 1974; Hughes. 
1982). 
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New behavior and sldlls leqnixed for coUecting and organizing achoolwide 
data and publidy sharing the results has strained some school social systems and 
woited to inhibit instead of foster oollegiality. It is one thing to ask a teacher to 
serve as researcher in hisAier own classroom and use the experience garnered to 
improve private practice^ it is something else to ask all on-site to marshall their 
action in support of common goals, legulariy monitor progress on these goals, 
publicly share results, and redirect/modify actions individually, as a giade level 
unit, or as a school based on assessment of effects. 

The following study describes the progress made thus far on increasing the 
variety, collection, and use of data in the 24 League schools during 1990-1991. 



Procedures 

This paper is part of a ku-ger effort to assess the progress of the League on 
its three major goals. The intent is to use the resulta of these assessmenta 1) to 
select the content for future meetings and the nature and types of information, 
examples, and processes selected for sharing across League schools and 2) to better 
understand the nature, difficulties, and possibilities of long distance collaboration 
between schools and universities in service of school improvement. 



This study was designed to explore several major questions related to 
participation in the League: 1) Do schools increase their use of existing data? 2) 
Do schools begin to look more closely at student achievement data as a 
determinant of progress on their schoolwide instructional initiatives? 3) Do 
schools broaden their data collection base beyond standardized teste to provide 
more accurate/authentic assessment of the effect of their schoolwide instructional 
initiatives on student learning? 4) Do assessment activities become more focused 
on what studente are experiencing academically and socially during instruction and 
on what they are experiencing as members of their school's institutional culture? 



Samplfi 

The twenty-four schools whose action plans and plaiming surveys form the 
unite of amdysis were members of the League of Professbnal Schools for 1990- 
1991. Producte developed by school teams as part of their school improvement 
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plans or as aids to reflection on the planning process provided the bulk of the data 
analysed in this study. Action pUms were collected in January/Februaiy 1990 from 
24 schools, in May 1990 from 24 schools, and in February 1991 from 21 schools. 
Planning surveys were collected from school teams in January/February 1990 and 
again in February 1991. Data reported here from other sources include results of 
a survey of changes in assessment practices administered in January 1991 and a 
summary of what school teams requested as content for future t« ffftt <"flff after the 
October 1990 annual conference and the February 1991 planning meetings. 

In their letter of application for membership in the League, these schools 
signed a c ommitment to collect data to assess progress on their initiative(s). 
Accqptance into the League is contingent upon having 80% of the faculty vote by 
secret ballot to join the League. This application or conmiitment letter specifies 
activities League staff agree to provide and activities schools agree to pursue, e.g. 
shared governance, instructional initiatives, and action research. League schools 
also agree to share individuaUy their experiences with colleagues and to share 
collectively the experiences of League schools through presentations and 
publications. 

The six questions used in each of the three sets of action plans, the concerns 
question and inhibiting / enhancing factors question asked in the two planning 
surveys, the changes in assessment question asked of the school contact person, 
and the general discussion question asked of each school team at the end of the 
October annual conference and the February planning meetings are listed bdow in 
Figure 1. School teams composed of a minimum of four persons, which must 
include the school principal and at least two teachers, completed the action phms 
together as part of a one or two day meeting of the League of Professional 
Schools. The action plan is a group product. The planning survey, which includes 
the concerns question and the question on enhancing / iohibiting factors in school 
change, is an individual product completed by each school team member. The 
question used in the survey on changes in assessment practice was mailed to and 
responded to by the school-designated contact person at each school site. And, 
finaUy, the discussion question responses summarized as part of this report are 
from one of three items listed on the Discussion Questions form used during the 
flnal one-half hour of all major meetings. League staff and the League's governing 
board use the information from these discussion questions in setting priorities and 
planning future activities. 
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Figure 1: Data Sources and Conteit Units Used 



1. What is the greatest priority for in8tru(;tional improvement 

tbat the leadership team can act upon this year and continue with 
over several years? 

2. What will be the hoped for results of such an initiative? 

3. What specific activities are neoessaiy for carrying out the initiatives? 
When wiU th^ be done? 

4. Who will be responsibh for which activities? 

5. What assistance or resources will be needed? 

6. How will progress and results be assessed?* 



Planning Siirvny fnr limgiiis of Prnfg«rionft1 jSchnnla 

1. Name the major instructional changes that have been made in your 
school in the psst three years and describe how they came about 

2. Describe the instructional dedsion-maldng process in your school. 

3. When you think about school-based teadership, what are you most 
concerned about? Do not say what you think others are concerned 
about, but only what concerns you now. Please write in complete 
sentences, and please be frank. 

4. What are enhancing and impeding factors that are important to consider 
before your school makes further changes?* 
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AMCwmfint-Siioffiy 



1. Has your school made any changes in assessment as a result of your 
participation in the League? 

CHECK ONE 

Y ES ^NO 

If yes, what changes have heen made? 



f^glift of Pmr««inn«1 Schools- ntacinMann QiMHrfmfi» 

1. What topics and workshops activities need to be on the program at 

our Winter Planning Meeting? (Changed to Spring Planning Meeting for 
Februaiy 1991 administration.)* 

2. What are your suggestions for our next edition of In Sitgn? 

3. Comments or just anything that you wish to express about services, 
future meetings, etc. 



* Indicates items within the context units from which referential units 
wens recorded. 



A brief description of the demographics of the schools that comprised the 
1990 League and were, therefore, the daU sources for this information are 
provided next as useful background reading prior to the analyses and results. The 
1990 Georgia League of Professional Schools had 24 members: 16 elementary 
schools, 3 middle schools, and 5 high schools. Of these, 2 schools were in urban 
settings, 16 in suburban, and 6 in rural. For socioeconomic status based on 
percenUge of free and reduced lunches, 11 schools had 10% or fewer students on 
free and reduced lunch; 3 had 11%-2S%: 8 had 26%-S0%:and 2 had over 50%. 
Minority populations ranged from 11 schools with less than 10% minority student 
enroUment; 5 witii 1 1% - 25%; 5 witii 26% - 50%; and 3 witii over 50%. In size of 
population of elementary schools, 1 elementary school had less than 250 students; 
5 had between 250 and 500; 5 had between 500 and 800; and 5 had between 800 
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and 12S0. The three middle schools were all in the 900 - 1050 student population 
range, and the five high schools were from smallest to largest: 390, 680, 900, 1200, 
and 18S0. Regionally, the 1990 League included schools from north, oentral, and 
south Georgia; however, the minority nspresenUtion (14 schools) was suburban 
schools within a 50 - mile radius of the AtlanU metropolitan area. 



Deffinitiona and Methoria of Analygin 

ivfinitionn. Figure 2 illustrates how schools are asked to use data, and 
Figure 3 illustrates eiamples of data sources school teams are encouraged to 
consider in their planning.(See Figures 2 and 3 on the following page.) The next 
four paragraphs define terms that were of critical value in analyzing the referential 
units and answering the four m^r questions on data use. 

RTigtiniy data are souToes such ss attendance figures, retentions, discipline 
referrals, suspensions, course grades, standardized test scores - all those "facts" on 
student and adult life that are readily available for analysis but may not have been 
organized for use or study. 

Sfiidimt mchi^vfutwnt data are Criterion referenced and norm 
referenced test scores, grades, retentions, and promotions. Usually, but not always, 
these are a subset of existing data sources that identify a student's status in the 
school's academic program. 

Rrnadgfiing hfjnnA atandardiaftd anliievMnent teata refers tO any measure of 

student progress cited in the action plan other than standardized criterion- 
referenced or norm-referenced test scores. 

Agawwfnent fncii«>f»d n« gfii<if»p»Q» arjidi>min nftrinl <>^ppri^t|frffi dl'""g 
ifMrtniction nr on ttigir rnle a« a mgmhftr nf thrnr adinnl ciilturR indudes meaSUTBS 

that focus on student attitude, academic or social self-concept, transfer of learning 
beyond the classroom, social skills, physical fitness, sense of efficacy, response to 
instruction, or role as school community member. 
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Content analysis with results displayed in frequency distributions are used to 
share the answeis to the four questions on data use. Content analysis allows one to 
make replicable and valid inferences from data to their contest. As a research tool 
it can "provide knowledge, new insights, a representation of (facts,) and a practical 
guide to action" (Krippendorf, p. 21.) Briefly, for answering the questions in this 
study, the physical contest units were the three sets of action plans and the two 
sets of planning surveys. The recording unit, using referential units, was any 
reference-word, phrase, sentence-*to data use, or data used, assessment of 
progress, or evaluation in Item 6 of the action plans and Items 3 and 4 of the 
planning surveys. 

The three sets of school action plans and the two sets of 
school-team-member planning surveys were read. All references to data use or data 
used in the items described above were recorded. Then each question became the 
stimulus for surveying each recording unit for answen to changes in data 
collection for school improvement. Recorded units may be coded as responses to 
more than one question. For eiample, standardized test scores may be coded under 
Question 1 on existing data; they would also be coded under Question 2 on use of 
student achievement data. So, while some items in Question 2 could be a subset of 
Question 1, use of any data sources-existing, conventional, or creative-focused on 
student achievement was of vital interest and tracked as a separate category. 

Planning Survey and Ch»nff>n M«Hi> rinriny thi> An«1y«« v>hu^ Qriginally,the 
concerns question was the only item on the Plannitu S wvey for tha f ^yin» of 
Profcmional Sfihools identified for analysis as part of this study. However, after 
reading all surveys in their entirety, the item on factors which enhance or inhibit 
change was added because it was in response to this item that approximately 
one-half of the comments about assessment were expressed. So, in order not to lose 
this information and to be able to consider it for analysis, these responses tiwc-atw*^ 
part of the pool from which references to data use were extracted. 

The concerns question used was modeled on the standard format for open- 
ended statements of concern developed by Fuller and her coUeagues and used in 
researeh on change by Neulove and HaU (Hall ft Hord, 1987). According to Hall, 
George, and Rutherford (dted in Hall ft Hord, 1987, p. 59), as respondents 
consider issues such as school-based leadership, they mentally "explore ways, 
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meuis, potential barrien, possible actions, risks, and rewards" in relation to the 
issue. This "aroused state of personal feelings and thought" about an issue as it is 
pefodved is rnncf^m Analysis of these responses in relation to asse ssment 
provided some infonnation about the kvd of awareness regarding assess ment 
issues, the level of implementation of data use as a guide to action, and topics or 
issues for future League meetings or newsletters. 



Results 

1. Do achooln incmnae thftir mm of existing data7 

Table 1 



Number of League Schools Citing Use of Existing Data 



Dates of Jan/Feb 1990 May 1990 Feb 1991 
Action 

Plans N=24 N=24 N=:21 

Schools 8 14 12 

Total 14 30 16 
Citations 



In citing ousting data for measuring progress and assessing re8ult»~such as 
failure rate, discipline referrals, test scores- League school teams moved from one 
third of all schools reporting use of information available in school records, 
folders, archives, and the like, in Januaiy/February 1990 to one-half or more of all 
schools reporting collection and organization of data from these sources in May 
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199C< ind February 1991. The most commoiUy cited sources of data were 
staiu'iardized test scores and student failure rates. 




it ifiitiativc? 



Table 2 



Number of League Schools Citing Use of Student Achievement Data 



Dates of 

Action 

Plans 



Jan/Feb 1990 
N=24 



May, 1990 
N=24 



Feb 1991 
N=21 



Schools 



8 



13 



12 



The number of school teams specifically describing measurement of some 
aspect of student achievement as part of their instructional initiative action plan 
moved from one-third of the schools reporting to one-half and holding. However, 
there were no referenu r to disaggregation of data foi\by different populations. 
There was some movement— from zero references in the January/February 1990 
plans to four references in May 1990 and four in February 1991— to looldng at 
items such as failure rates both schoolwide and classroom by cUusroom; 
comparing changes in failure rates at set times; organizing data from teacher 
records to look at pre and post reading levels of students after a change in 
language arts instruction; and team assessment of subject-area grades (A, B, C, D, 
F) by grade level. 
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Tables 



Number of Schools Citing Data Used Other than Standardized Tests 



Dates of Jan/Peb 1990 May 1990 Feb 1991 

Action N=24 N=24 N=21 
Plans 

Schools 5 7 8 
Total 

Citations 10 22 19 



The number of school teams describing collection of information beyond 
achievement test scores to items such as increased amount of student writing, 
increase in number of books read, writing portfolios and to existing data such as 
reading records, tracking of retentions, and analysis of student grades has 
incT^eased slowly. Few schools have moved into careful scrutiny of instructional or 
c.!rricular goals and made plans for expanding the accuracy of measuring progress 
toward these goals, even when the identified priority seemed to demand such an 
expansion, e.g., as in the following instructional ioitiatives: 

- Broaden participation in and focus on personalized learning in language 
arts and science. 

- Assist students in taking greater responsibility for their behavior and 
learning. 

- Improve student performance by focusing on reading in the content areas 
for enrichment and enjoyment 

16 
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Table 4 



Number of Schools Citing Assessment of Students' Responses or 
Attitudes to Social, IntellectualXAcademic, or Physical Experiences 
Occurring during School 



Dates of Jan/Feb 1990 May 1990 Feb 1991 

Action N=24 N=24 N=21 
Plans 

Schools S 9 10 
Total 

CiUtions S 11 12 



The number of schools describing collection of information on what 
students are experiencing during their days within the confines of the school- 
beyond academic achievement measures such as grades and test scores or fate 
control measures such as referrals, suspension, and retentions,— has increased 
during this first year. Items cited on the first set of action plans were four student 
surveys and one reference to tracking amount of student work completed and 
turned in. While six of the eleven assessment activities cited in the May action 
plans were "student surveys," five references moved beyond this general description 
of an assessment instrument to items such as "determine whether there are 
significantiy fewer schedule change requests made after the [new transition / 
advisement] process has been implemented"; implement follow-up on students who 
graduate; and in an elaborated description of the content of surveys, such as 
"administer student questionnaire on the effectiveness of the advisement program." 
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III tbe Febnuury 1991 action plans, several conventional assetfsment sources 
were dted, such as reading interest inventories, home surv^ of student's reading 
habits, suiv^ about the learning process students were eiperiendng, and self- 
analysis comparing individual to school/class baseline data (Where am 1 in this 
picture?). However, conventional and creative sources-such as student interviews, 
observations, videotapes of changes in cooperative learning activities, production 
of artifacts students choose to produce-to gather information were not dted. 

rnnci>rn« and Tmpedifitf Pactftftt 

In January/February 1990 and again in February 1991, individual school 
team members responded to four questions on the PUnwiny Si irvisy far thf. f ^gii#i 
of Preffiwrional Schools- Along with their responses, team members provided their 
position in the school. Using questions three and four of the planning survey as 
the content unit, each of these surveys was read, and all references to evaluation, 
assessment, data use, or data used were recorded. 

For January/February 1990, only four respondents expressed concern 
directly and deariy related to assessment of the goals or actions implemented by 
the school community. Four other respondents dted concerns or inhibiting factors 
possibly related to assessment. For example, directiy rdated responses were: 

Elem. Techr. (I) Will these conmion goals be evaluated? 

Asst. Prin. (C) I am concerned about allowing teachers not only to voice 

opinions, but to plan, implement, and evaluate some of the 
strategies they feel work. 

Asst. Prin. (C) Do we use research from the field and data gathered on 

our students? 

Prin. (C) Keeping people motivated to continue to assess what they 

are doing and why. 
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The fonr statements that eiinessed s peripheral idatioiiship to action 
leseazch or assessment of progross were: 



Asst. Prin. 



8th Grade 
Teacher 



Principal 



6th Grade 
Teacher 



(Q I am very troubled about emphasis on standardized test 
scores. It seems that we try to teach the test to aet high 
scores to "beat' their school's qfstems. Our teachen 
complain about lack of fieedom because of GTOI [Georgia 
Teacher Observation Instrument] and test score emphasis. 

(C) Also, each dass is *doing its own thing* and I worry 

about measuring up or my kids not getting what they need 
for testing or for preparation neat year. 

(I) Impeding factor - evaluating where we are at this point in 
this shared leadership. Evaluating what direction the 
psrents want the school to go. 

(1) Need to consider . . . effect on students (what's best for 
them). 



In the February 1991 surveys, twenty-three respondents identified specific 
concerns about assessing the efiect of their initiative or about using data to 
measure the effects of changes on students. The dominant pattern in these 
responses (16 of the 23) was concern about the need to assess progress on the 
school improvement initiative. Examples of responses follow: 



Elem. 
Tescher 



(Q Evaluation also concerns me — we must be able to show 
ourselves and others that our work was worthwhile. 



3rd Grade 
Teacher 

Elem. Teach. 



4th Grade 
Teacher 



(Q I am concerned with the asfessment of whole language 
lessons and interdisciplinr j* ' unit teaching. 

(Q I would like to work more on gathering data to evaluate 
progress. 

(I) Need data collected on the good things we've done. 



Instructional (I) Additional action research needs to take place and data 
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Lead Teacher 



3rdGtade 
Teacher 

Sth Grade 
Teacher 

Central OfT. 



gathered to see where we are with our instnictioiial 
initiatives. 

(I) We are working on evaluation ideas. We need to know if 
what we are doing is best for children. 

(Q We are still grappling with the idea of what data do we 
need to look at and how do we need to look at it. 

(C) It will be difficult to make the time to complete aU the 
items needed - - especially action research. Some teachers may 
not be willing to take responsibility for solving concerns 
that arise during action research. 

Major diCTeiences between the two samples were the increases from 1990 to 
1991 in number of respondents expressing concern or awareness of the need to 
collect and use data to assess progress on their initiative (January/February 1990 - 
8 of 123 re8pondent8,7%: February 1991 - 23 of 104, 22%). Responses had a tighter 
link semantically between actions being taken or implemented and use of data to 
determine the effectiveness of these actions. For example: 

Elem. (I) We need to monitor and evaluate our current changes. 
Teacher 

Principal (I) We need ... to decide on useful ways to assess what we are 
doing. 

2nd Grade (I) We have our goals set and have even made plans and 
Teacher implemented some. What we need to do is assess our progtess. 

4th Grade (C) Vm also concerned about how implementation of the different 
Teacher focuses are monitored after the PSI Team has approved a 

particular idea brouglht before it by a task force. 



Another major difference between the 1990 and 1991 planning survey 
responses had to do with the positions or roles of respondenta citing concerns 
about assessment of progress. In the January/February 1990 data, only three of the 
eight respondenta were teachers; the other five respondenta were principals and 
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assastant prindpils. Teachocs wsie imder-repiesented by role in tiidr number of 
responses: in the total group, 81(66%) of the 123 respondents were teachers and 
42(34%) were principals, assistant piindpals, and central oflfice supervisors. In the 
February 1991 data, 17 of the 23 respondento were dassroom teachers; this 
representation was more proportionate to their role representation in the total 
population of respondents. The percentaBe of chuHffoom teachers expressing 
concerns about assess ment of progress moved from 4% of the total set of 
respondents in 1990 to 23% of the total set of respondents in 1991. 

Responses to the rhanyen in Anamnmni Qiii>«tinn 

Eight of the twenty-four League school contact persons responded "Yes," 
our school has made changes in assessment as a result of partidpating in the 
League of Professional Schools. In their description of changes, four of them dted 
administering student attitude or self-esteem surveys, two dted school climate 
surveys, and two dted parent surveys. One contact person described "rudimentary 
first steps" as changes made, for example: 

We have begun to get some more teachers' attention about assessment 
and how important it is. We have begun some dialogue about how we assess 
students in our classrooms and what kinds of information teachers need to 
collect to be able to assess a student's performance. 

Two contact persons aUio described securing informal assessments such as 
feedback from their School Improvement Team, from grade levd planning 
meetingB, and from their across-the-grade-planning meetings. One contact person 
described a more thorough analysis of results from the GfificgiajCotfidfUL 
RcferencBdJjcata in reading, writing, and mathematics and from the state-required 

Iowa Teste of Bmib Rlrtll« 



Responses to nincnsBinn Queatinna 

The number of school team requests for future workshop sessions and 
activities on assessment increased each time this discussion question was asked: 
"What topics and workshop activities need to be on the program at our [next] 
meeting!?" The sharpest increase was from the October 1990 meeting to the 
Februaiy 1991 meeting. 
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Asaessment or action leseaich began as a low-interest topic but gradually 
became a prominent concern among League schools. At the end of the October 
meeting* eight school teams requested additional assistance and activities focused 
on the action research process, on case study examples of assessment, and on 
a ssi s t a nc e in developing assessment plans. At the conclusion of the February 1991 
meeting, fifteen school teams requested additional activities on action research. 
Specific items mentioned were to continue the progression of sessions on this topic, 
to continue providing examples and group work-time on assessment, to continue 
the sharing from school teams on what had been done in those schools farthevest 
along in the action research process, and to provide more concrete assistance on 
developing assessment plans. In terms of assessment plans, teams asked for hdp in 
determining who should collect data, when it should be collected, how or what 
instruments to use, how to design effective data gathering instruments, and where 
to find good computer programs to use for organizing data. 



League Activities to Promote Action Research 

What was done to help these 1990 League schools collect and use data for 
school improvement? Here is a brief chronological outline of key activities: 

JamMry/Pghruary 1QQI 

As part of the two-day orientation and planning workshop prior to their 
affiliation with the League, school teams were provided with a brief (IS min) 
overview of action research. After the overview, team members were asked to 1) 
individually answer, 2) discuss as a group, and 3) come up with a team response to 
three questions on sources of information used to assess the effects of instructional 
programs on students. 

These three questions were: 

(1) What data do we use? 

(2) What other data— available and/or must be sought- could we use? 

(3) What information would help us set schoolwide instructional 
goals? 
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jMUAry/PfthniAry 1QQ0 

The last two houis of the orientition and plamung meeting had school 
teams developing their action pUus. These plans indade a section on assessment of 
progress. During this workshop planning time, four to six League staff members 
weie available on request to provide immediate planning assistance. 



Maroh 1900 - March 1QQ1 

Telephone consultation was available with League staff about data 
collection, instrumentation, organization of data, analysis of results, and general 
trouble-shooting. 



At the fiist Spring Planning Meeting, participants could attend a 55 minute 
session on assessing progress. Each school team was encouraged to send at least 
one lepiesentative to this bieikout session. Participants were provided with two 
resources to support their data collection efforts: 

1. Smormbord for Baadine nfttaVFviHmice of ProgrBHrinn nn T mrtnictinnal 

Tnitiativm. This eleven page packet listed over a hundred data collection activities 
generated by League staff after they had studied the action plans written by the 24 
League schools. Samples of potential activities included existing data sources such 
as retention rate/ promotion rate by school, by grade level, by teacher, and by 
particular groups ( male/female, hlaickJ white, high/mid/low SES) and the 
percentage of students participating in school-sponsored organizations like DECA, 
PBLA, Student Council, Yearbook Club, Drama Club by school, grade level, 
ethnic group, academic track, etc. Conventional data sources such as number of 
books read by students recorded individually, by classroom, by grade level, and 
by school; library use by function and by number of books checked out, by grade 
level, by school; writing samples; student attitude surveys in the content areas 
followed-up with interviews; nature of assistance, numbers of, and/or amount of 
time contributed by volunteer parents or community members. More creative 
sources reco mmend e d as possibilities included videotapes of students working in 
cooperative groups, in-d^th case studies on a sample of students, student 
evaluation of instructional materials used, and portfolios of student work. 
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2. Antmn um^rrh KntelvMilr In iditition to this lisUng of posubiUUcs, 
each school was provided with an Action Research Notebook that included 
descriptions of, scoring guides for, and in many cases actual copies of 23 
instruments. League staff had secured prior permission for schools to copy many 
of these instruments; when copying was not permitted, ordeiing information was 
included. The four sections under which these instruments were grouped follow: 
(A) Questionnmires and Surveys - Attitude (for students and teachers); (B) 
Innovation Concerns (for school stafi); Locus of ControlVConceptual Level; and 
(D) School Climate. 

The May session concluded with a brief application activity. Participants 
were urged to consider these materials and suggestions later in the day when they 
would be working with their school team members on their school's action plan. 

October 1990 

At the annual conference, participants were provided with another 45 
minute session on assessing instructional initiatives. This session had three parts: 
lessons from the field, assessing implementation results, and shaiing from schools. 
During lessons from the field. League staff provided tips for getting started with 
data collection, for selecting methods and instruments, for reporting results, and 
for making adjustments. In assessing implementation and results, considerations 
in and examples of assessing implementation were dtscuMtd, including levels of use 
and appropriateness of use. Assessment of utilization of components of the school 
program--curriculum development activities, stafiT development activities, 
evaluation activities, and adminisUative prooedures-to support implementation 
was also discussed. Two other points presented and emphasized through ezanqiles 
were the need to assess the effects of major initiatives on all stakehold^s— 
students, teachers, administrators, parents, community— and the value of using 
multiple data sources to provide infoimatioo. The emphasis was on going beyond 
standardized test scores to student products, student performance, observations of 
desired actions, and simply counting incidences of the behavior under study. As in 
the May session on assessing progress, school teams were encouraged to count 
behaviors and products relevant to their initiative and organize their data into 
simple frequency distribution tables. For example, dq)ending on the goal(s) of the 
action plan, school faculty could count items such as discipline referrals, number 
of books read, number of and amount of time contributed by volunteer parents or 
community members, number of simulations each teacher had tried as part of the 
new social studies curriculum. 
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During acliool sharing time, ptrtidpants were asked what was needed next 
to support their action research efforts. The primary responses were 1) start over 
with concrete steps on how to get started; 2) let's hear from some schools that 
have made progress on action research; and 3)provide longer sessions with more 
work time on this topic. A comment from one of the principal's attending this 
session seemed to summarize the n^jority feeling succinctly: "We are just 
beginning to understand why we need to do somu of the assessment things you 
were talking about last spring [May 1990]. Start over and go step-by-step." 

Two other SS minute sessions on methodology or sources of data were 
offered as part of the annual conference. One was on observation of student work, 
and the other was on interviiBw techniques for use withNby staff, students, or 
community members. The emphasis in each of these breakout sessions was on 
using these techniques to enhance understanding of what individuals were 
experiencing and to assess progress on school improvement initiatives. 

Pchniftfy 1QQ1 

The Winter Planning Meeting breakout session on assessing progress was a 
two and one-quarter hour session. For an hour and a quarter, three League 
schools whose action plans and conversations had indicated they were collecting 
data relevant to their instructional initiatives presented. School team members 
shared what had been done at their schools, provided copies of results illustrating 
how they had organized their data, and provided brief written overviews of their 
action research efforts. The elementary school presenters focused on how tlie> had 
involved teachers schoolwide in thinking about assessment, and the middle school 
and high school presenters focused on their collection of baseline data. Session 
participants questioned presenters about how and should classroom-level (teacher- 
levd) data be shared schoolwide, such as number of biudents failing English 11 A, 
and what actions were being taken to achieve the goal(8) of each school's 
initiative. 

The final hour of the session inluded a discussion of Figures 2 and 3 as 
operational schema to use in planning and conducting action research. While the 
same content had been delivered in earlier presenUtions, materials, and handouts, 
these two figures had just been developed to use with the 1991 potential League 
schools during the January orientation and planning workshops. Based on League 
schools' progress in this area, staff were searching for a clear, accurate conceptual 
representation of the action research cycle and of the broadening of data sources 
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emphasis as a fnunework to shave with new schools duxing the initial picsentation 
of this topic. Session participants were asked to use the two figures for self- 
evaluation of their school's action research efforts. 

Eiamples of data handling, both oral and written, were provided for each 
step of the action researoh cycle: collection, oiganization, analysis, intecpretation, 
and action. (See Figure 2.) Examples or descriptions of collection andNor use of 
existing, conventional, and creative sources were provided also. (See Figure 3.) 
The final 20 minutes of the session was used for an application activity. Each 
school team member was asked to consider hisMier schoolwide initiative and write 
a list of assessment activity recommendations to take back to the full planning 
team session after limch. 

FehruMTy\M>rdi\April 1091 

Schools received one full-day on-site visit by a League staff member. The 
primary intent of this visit was 1) to provide school teams with a status report on 
their shaved governance process, and 2) to provide League staff an opportunity to 
study variations of shaved governance in use by different faculties in 24 school 
sites. During this visit, one-half of the day (3 1\2 to 4 1\2 hours) was spent 
interviewing staff members- both those on the school governance leadership team 
and those who were not— students, and the school principal. Any materials 
developed by the school staff related to the three League emphases (shared 
governance, enhanced education through instruction, action research) were 
collected, reviewed, and phu:ed in the school file. During the second half of the 
day. League staff fulfilled school requests for service such as assisting school teams 
by participating in trouble-shooting sessions; facilitating planning meetings; 
conducting staff development sessions in content areas; presenting shaved 
governance or action vesearoh overviews during school faculty meetings; etc. 

Within two weeks of the on-site visit, the school contact person is mailed a 
status report that addresses shared governance in X School; and, dq>ending on the 
on-site facilitator, this report may include sections on the school's instructional 
initiative and on action research at X School. 

Reflections on Action Research in the League 

Within th V League of Professional Schools, shared governance developed 
more rapidly than did the action research component. This was true of all sites. 
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At this point nearly bU sites have colleagial discussions about school improvement; 
most have plans that guide action; but none has a well-developed action research 
process. 

However, many of the sites appear, after a year of involvement, to be eager 
to study action research and how to conduct it. Also, what appeared at first to be 
reluctance to engage in action research in some sites appears now to have 
diminished. When implementing a complex process such as using action research to 
guide collective action for school improvement, school faculties may need a year of 
activities, examples, and discussion to develop readiness for implementation. 

Until we have further experience with the efforts of League members, it will 
be difficult to project patterns of need for support or exactly how to provide 
sufficient technical support for action research through our school-university 
collaborative framework. However, based on this year's experience with twenty- 
four widely-different schools, many school teams appeared to need to learn about 
the kinds of assessment devices that are available and how they can be used. Early 
formulations of assessment goals were either too general to be acted on (use of 
surveys sui generis, rather than specific surveys) or seemed to identify common 
instruments such as standardized tests more or less "because they were there** than 
because they fit the need or could provide information in the time frame needed 
for problem solving. Very few sites have become, at this point, oriented toward 
the use of some of the assessment sources that are readily available on-site— such 
as grades, promotions, referrals, disciplinary actions, or student opinion about 
problems and successes. After another year's experience conducting action research 
on the League as a support unit for school improvement and studying the efforts 
of League schools, we may know more about the developmental aspects of action 
research for school improvement. 

A Perannal Note 

Although I prefer in general to stay close to the ground of the data, I would 
like to share a few personal observations or impressions which I cannot assert with 
the same sense of data-groundedncss that I can that elements of shared governance 
appeared more quickly in our League schools than did the process of action 
research for school improvement. 

However, ... it appears to me: 
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That the public sharing of data has a confrontational connotation to some 
of our teachers and administrators. Some sites have reported that problem- 
identification linked to specific personnel is very uncomfortable and difficult for 
them to handle productively. (The number of kids failing algebra is five times 
those failing calculus. One school team is responsible for forty percent of our 
failures schoolwide.) 

Some types of confrontation can be beneficial, however, as I observed 
occasionally. One faculty was shocked when they counted the nuiaber of children 
who were failing the ninth grade and the number who had traditionally dropped 
out. Recovering from the shock, they responded by taking action. I observed 
other faculties where the same type of pattern occurred-data collection, shock at 
the evidence, and (not without struggle) redirection of their initiatives. 



I think that it may be necessar} for schools to experience this kind of 
confrontation if school improvement is to become serious. Schaefer's vision of the 
school as a center of inquiry-still one of the best statements of collective action by 
enipowered professionals-requires that faculties become realistic in describing 
current conditions, scholarly in postulating courses of action, and realistic again in 
assessing the consequences. All of these require the ability to deal with 
information that confronts our assumptions. And in our society, confrontation, 
whether social or intellectual, is no trivial matter. The "research" dimension of 
action research requires that we learn to seek data that will challenge as well as 
confirm, that we set courses of action not as sure solutions but as the best we 
know at this time, and that we understand that confrontation with data about how 
well our idea worked is, seen properly, self-enhancing and a spur to the next phase 
of action. 
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FIGURE 2 

Action Research: Data Use 
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FIGURE 3 

Action Research: Data Used 
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